Programme For International Student Assessment Research Articles

A comparative study of critical reading abilities among students in Malaysia and Vietnam: Insights from PISA-based assessment

This study conducted a comparative analysis of critical reading skills among engineering students at private universities in Malaysia and Vietnam, using the Programme for International Student Assessment (PISA) reading framework. Involving 182 undergraduates, the research included a quantitative critical reading test followed by statistical analysis (T-tests) to examine score variability among first-year and final-year students. The analysis revealed significant performance differences among first-year students but not among final-year students from both countries. To complement the quantitative data, semi-structured interviews with 16 participants across different critical reading levels were conducted. These interviews aimed to capture students’ perspectives on the PISA test, highlighting challenges such as the Western orientation of texts, English vocabulary, and the format of open-ended questions. Integrating quantitative and qualitative methods, the study offers an in-depth understanding of the critical reading abilities of engineering students in Malaysia and Vietnam, underscoring the importance of considering cultural and linguistic factors in educational assessments.

Unlocking the secrets of students’ mathematical literacy to solve mathematical problems: A systematic literature review

Mathematical literacy is the ability to use mathematical knowledge in real-life situations, making it an essential component of education because of its importance in solving everyday problems. Mathematical literacy is also part of the Program for International Student Assessment (PISA) global assessment. Because of the importance of the subject, this Systematic Literature Review (SLR) investigated the relationship between students’ mathematical literacy and their ability to solve mathematical problems. This SLR uses the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) approach was used, and articles published from January 2013 to August 2023 were obtained from databases including ERIC (342), ProQuest (1,329), and Scopus (27). Following PRISMA, a total of 20 articles were included in the review. Of the 20 articles, most were conducted on junior high school students in Turkey. The majority of reviewed studies found students to have a low level of mathematical literacy, which caused difficulties in formulating problems. The examined studies also revealed several internal and external factors affecting mathematical literacy. Problems used by PISA were the most frequently employed to measure students’ mathematical literacy in the reviewed studies.

Re‐examining the relationship between patience, risk‐taking, and human capital investment across countries*

SummaryHanushek et al. (2022) show that students in countries in which people are more patient and less risk‐taking perform better in the Programme for International Student Assessment (PISA) test. In this paper, we probe the robustness of this study. Our narrow replication shows that most of the results are robust to alternative model specifications. Our broad replication shows that the main results are robust to measuring student performance with data from the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) instead of PISA.

Forms of Peer Victimization and School Adjustment Among Japanese Adolescents: A Multilevel Analysis.

The psychosocial correlates and consequences of peer victimization are well documented. However, there is limited knowledge about whether different forms of peer victimization (relational and physical) are predictive of school-based social and motivational factors among adolescents from non-Western cultures. The present study examined the relationship between individual and school-level forms of peer victimization and school adjustment among Japanese adolescents, and the mediating role that these factors may play. The Japanese sample (N = 6109 from 185 schools, Mage = 15.78, SD = 0.29, 51% girls and 49% boys) was drawn from a large international dataset, the Organization for Economic Cooperation and Development (OECD) Programme for International Student Assessment (PISA) 2018. Results showed that school-level relational victimization was associated with individual-level relational victimization, and school-level physical victimization was associated with individual-level physical victimization, after controlling for age, gender, and socioeconomic status. Individual-level relational victimization was also uniquely associated with indices of school adjustment (negative affect, positive affect, and fear of failure) over and above physical victimization. While controlling for relational victimization, individual-level physical victimization was associated with indices of school adjustment (positive affect and meaning in life). In further findings, school-level relational and physical victimization were indirectly, but not directly, related to some of students' school adjustment through individual-level relational and physical victimization. These parallel and differential associations suggest the importance of considering the role of relational and physical victimization in school adjustment among Japanese adolescents.

Diagnostic Classification Models for Testlets: Methods and Theory.

Diagnostic classification models (DCMs) have seen wide applications in educational and psychological measurement, especially in formative assessment. DCMs in the presence of testlets have been studied in recent literature. A key ingredient in the statistical modeling and analysis of testlet-based DCMs is the superposition of two latent structures, the attribute profile and the testlet effect. This paper extends the standard testlet DINA (T-DINA) model to accommodate the potential correlation between the two latent structures. Model identifiability is studied and a set of sufficient conditions are proposed. As a byproduct, the identifiability of the standard T-DINA is also established. The proposed model is applied to a dataset from the 2015 Programme for International Student Assessment. Comparisons are made with DINA and T-DINA, showing that there is substantial improvement in terms of the goodness of fit. Simulations are conducted to assess the performance of the new method under various settings.

The Importance of Context in a Global World

Global education is often framed in terms of standardised testing that makes comparisons across nations. This is particularly evident with international measures like the Programme for International Student Assessment (PISA), which tests 15-year-olds in member countries. Images on the PISA website provide representations of education that seem to clash with some of the contexts where we have conducted research. This prompted an investigation into educators’ talk about local contextual realities, and how—or whether—they were impacted by global calls for quality education. The study focused on three different educational programs: education for adolescent girls in a refugee camp in the Kurdistan Region of Iraq, school education in a western Queensland rural/remote town in Australia, and the Migrant Education Program in rural Illinois in the United States. The data were collected through semi-structured interviews and field observations. Following data analysis, three case study narratives were constructed. A final step of analysis identified four shared pillars that made learning in rural and remote areas impactful and effective: context relevance, educators’ openness to learning, flexibility of teaching/learning approaches, and responsiveness to learners’ needs. The data demonstrated that the educators had a broader picture—national or global—in mind, but their main aim was to provide learning opportunities that were responsive, flexible and contextually appropriate to their location.

Age of entry into early childhood education and care, literacy and reduction of educational inequality in Nordic countries

ABSTRACT An early start to good-quality early childhood education and care (ECEC) is considered beneficial, especially for disadvantaged children's development and educational outcomes. This assumption was tested using the latest two waves (2015 and 2018) of data from the Programme for International Student Assessment (PISA) in five countries using the Nordic model of early education and care: Denmark, Finland, Iceland, Norway, and Sweden. The article finds evidence of the overall positive association between the age of entry in ECEC and literacy at age 15 in all Nordic countries. However, the relationship is non-linear, and the highest benefits seem to occur following entry into ECEC from ages two to three. The link between family background and ECEC enrollment largely explains this association. We did not find that ECEC would generally compensate for low socioeconomic status (SES) in children’s achievement. However, the Matthew effect was observed in Norway, where an early ECEC start is more strongly associated with literacy scores for affluent children than disadvantaged children. These findings have limitations due to their correlational nature. Still, this article indicates that even in high-quality universal ECEC systems, early preschool education is not a panacea for lowering achievement gaps due to parental background.

Examining the relationship between online chatting and PISA reading literacy trends (2000-2018)

Strong claims have regularly been made in the popular media about the alleged negative effects of young people's language use in written exchanges via digital media (what is known as textese). This is especially believed to affect their reading and writing skills. However, the scientific research literature has hardly confirmed such effects. Its conclusions are generally based on cross-sectional comparisons between individuals who differ in how much they exchange written messages via digital media. In contrast, the present study examines to what extent per-country changes in the prevalence of online chatting coincide with changes in the reading literacy of consecutive student cohorts. This approach presents a novel perspective. The findings relate to data from the Programme for International Student Assessment (PISA) surveys conducted during 2000–2018. The data analysis showed a strong correlation between variations in per-country patterns regarding the spread of online chatting during 2000–2018 and changes in the reading literacy of 15-year-olds. Countries with low chatting prevalence in 2009 showed substantial improvement in reading literacy from 2000 to 2009. These trends typically reversed to declines when after 2009, online chatting prevalence sharply increased in those countries. A contrasting pattern applied to countries with high chatting prevalence in 2009 and decelerating growth in online chatting afterward.

Examining student, parent, and school factors predicting science achievement using a multilevel approach: the case of Hong Kong from the Program for International Student Assessment 2015

Examining student, parent, and school factors predicting science achievement using a multilevel approach: the case of Hong Kong from the Program for International Student Assessment 2015

Across four nations: Comparing the discourses of adolescents' digital literacy

AbstractIn this study, we investigated media reports and literacy research in four nations—China, Morocco, the Republic of (South) Korea, and the United States—about the relationship between adolescents' literacy and use of digital media, or digital literacy. We present short “snapshots” of adolescents' digital literacy in each country and then compare these to findings in a report about adolescent literacy and uses of digital media published by the Program for International Student Assessment (PISA). Our analysis indicates significant variation across countries in both literate traditions and adolescents' access to digital media, and notes that these interact to create unique conditions for adolescents' digital literacy in each country, even as, across the four nations, adolescents' capacity to innovate and solve problems with digital access seems constant. In conclusion, we are cautious about making global claims about the state of adolescents' literacy worldwide but point to important findings about how the use of the internet in schools seems to have a positive impact on reading performance and offer some implications for classroom practice.

Determining the factors affecting the ICT use of Turkish students attending PISA-2018: Multilevel profile analysis approach

AbstractThis study aims to classify student profiles based on the type and frequency of Information and Communication Technologies (ICT) usage. Each profile exhibits homogeneous characteristics and heterogeneous characteristics compared to other groups. Additionally, the study investigates whether covariates at the school and student levels create differences in these profiles. For this purpose, a mixed-method analysis (Latent Profile Analysis (LPA), Multi-level Latent Profile Analysis (MLPA) and Multi-level Logistic Regression Analysis (MLRA)) were used to identify ICT usage profiles. The study analysed data from 6,890 Turkish students across 186 schools participating in PISA 2018 (Program for International Student Assessment), revealing three distinct ICT usage profiles: Profile-1 (Low-Level ICT Users (19.06%)), Profile-2 (Intermediate-Level ICT Users (71.10%)), and Profile-3 (Advanced-Level ICT Users (9.84%)). The MLRA showed that students with low ICT usage have less access to ICT resources. Students in the Low-Level ICT Users profile also had lower scores on ICT-related motivation factors (interest, competence, social interaction, and autonomy). No significant effect of ICT usage on mathematics, reading, and science scores was observed. Variance differences at student and school levels could not be fully explained by covariates at the school level (type of school, ratio of computers connected to the internet per student, lack of educational materials and personnel). However, the ratio of computers per student effectively explained these differences. According to an MLPA, schools were categorised into two groups based on ICT resources: 'high' and 'low'. In schools with high ICT resources, the proportion of students in Profile-2 was found to be 45.8%, while those in Profile-3 accounted for 9%. In contrast, in schools with low ICT resources, the percentage of students in Profile-2 was 22.45%, and in Profile-3, it was 1.82%. These findings indicate that the level of a school's ICT resources significantly impacts students' technology usage skills. A significant finding of the research is that most Turkish students use ICT at an intermediate level at home for school lessons. The COVID-19 pandemic has further increased the importance of studying at home using ICT resources.

Typologies of secondary school student academic resilience in science with classroom and school context predictors

ABSTRACT The education system strives to help students from low-income families achieve academic success. Academic resilience is related to not only individuals but also classrooms and schools. This study aimed to construct a comprehensive resilience model in science domains that presents the image of resilient students and describes the mechanisms by which they arise. This goal was achieved through a parametric multilevel latent profile analysis with level 1 and level 2 covariates. Latent profile indicators included variables reflecting motivation, subjective well-being, and engagement in science of secondary school students. Variables related to science classroom and school contexts constituted the levels 1 and 2 covariates. We used secondary data from Japan and Macao (China) collected in the 2015 Programme of International Student Assessment (PISA). The sample sizes were 1594 and 1114, respectively. We identified three resilience levels among students. The emergence of resilient students benefited from science classroom discipline, teacher-directed instruction, inquiry-based teaching, adjusting instruction, and teacher support. In Japan, the most robust predictor was inquiry-based teaching, while in Macao, it was teacher-directed instruction. Therefore, science teachers play a crucial role in building students’ resilience. It also suggests that a small class decreases the likelihood of students entering the high-level resilience group.

Preferred Oral Corrective Feedback through Existing Internalized Language Prejudice of Students in the Philippines

Following the 2022 Program for International Student Assessment (PISA) test results, it was found out that the Philippines ranked 76th in reading comprehension among 81 PISA-participating countries. Students from the Philippines are still among the least proficient in the world in reading comprehension. With the country performing below the standards and criterion set by the organization, the researchers investigate internalized language prejudice as learners’ primary obstruction in acquiring the English language. Provided that the Oral Corrective Feedback (OCF) is essential for students’ learning process and performing separate functions in acquiring the English language, this paper aims to figure out their preferred OCF in acquiring the language. Furthermore, the proponents of the study delve into the reasons of the selected students when given an OCF in connection and relevance with how internalized language prejudice, stereotypes, and preconceived notions come into play in terms of the effectiveness of the English language learning and teaching. Lastly, the paper used  mixed methods in gathering data: quantitative through an online survey form and qualitative through Focus Group Discussions (FGD).

Internet Use Behavior and Adolescent Mental Health: The Mediating Effects of Self-Education Expectations and Parental Support.

This study focuses on how Internet use behavior affects adolescents' mental health and whether self-education expectations and parental support mediate the relationship between Internet use behavior and adolescents' mental health. The data for this paper came from the results of the student questionnaire of the 2018 Programme for International Student Assessment (PISA 2018), which was a structured questionnaire that asked students about their family situation, school life, studies, internet use, and mental health, among other things. A sample of 336,600 children in grades 7-13 was selected for this study. The data were analyzed using STATA version 16 and the theoretical framework was tested using a mediated effects model. The results of the study showed that Internet use behavior made a positive contribution to mental health and the mediating effects of self-education expectations and parental support on the relationship between Internet use behavior and adolescent mental health were all significant. It is recommended that appropriate policies should be formulated to help adolescents use the Internet rationally, and the positive effects of parental support and self-education expectations should be utilized.

Assessment of scientific literacy levels among secondary school students in Lebanon: Exploring gender-based differences

In recent years, educational reforms have made scientific literacy a major priority because of its increasing importance in today’s dynamic world. Program for international student assessment 2015 evaluated scientific literacy from 72 countries, and Lebanon ranked 65th. Scientific literacy levels among Lebanese secondary school students and gender differences are unknown. This study fills this research gap. A total of 130 students from Lebanon’s private secondary schools were involved in this research. Scientific literacy levels were assessed using an instrument developed by Gormally et al. (2012), who conceptualized nine science competencies contributing to scientific literacy including understanding research designs, creating and interpreting graphs, and solving quantitative problems. Welch’s ANOVA and Mann-Whitney U tests were used to analyze the data. Lebanese secondary school students have very low scientific literacy levels, and no significant difference was observed between genders. We present recommendations for adding desired skills to the curriculum based on a review of the underlying factors.

Understanding the disparities in Mathematics performance: An interpretability-based examination

Problem:Educational disparities in Mathematics performance are a persistent challenge. This study aims to unravel the complex factors contributing to these disparities among students internationally, with a focus on the interpretability of the contributing factors. Methodology:Utilizing data from the Programme for International Student Assessment (PISA), we conducted rigorous preprocessing and variable selection to prepare for applying binary classification interpretability models. These models were trained using the Stratified K-Fold technique to ensure balanced representation and assessed using six key metrics. Solution:By applying interpretability models such as Shapley Additive Explanations (SHAP) analysis, we identified critical factors impacting student performance, including reading accessibility, critical thinking skills, gender, and geographical location. Results:Our findings reveal significant disparities linked to resource availability, with students from lower socioeconomic backgrounds possessing fewer books and demonstrating lower performance in Mathematics. The geographical analysis highlighted regional educational disparities, with certain areas consistently underperforming in PISA assessments. Gender also emerged as a determinant, with females contributing differently to performance levels across the spectrum. Conclusion:The study provides insights into the multifaceted determinants of student Mathematics performance and suggests potential avenues for future research to explore global interpretability models and further investigate the socioeconomic, cultural, and educational factors at play.

Hunger Games

Undernutrition among children has always been a major concern in the Philippines. According to the Philippine 2018- 2019 Expanded National Nutrition Survey (ENNS) reported in 2022, the prevalence of underweight among children less than five years old was 18.8% among males and 19.3% among females. It was higher in the rural areas and highest among the poorest wealth quintiles.1 In terms of magnitude and severity, the prevalence was considered as a medium public health problem.
 In line with this, we note an article published in this issue of our journal that looked at the factors affecting survivalof children with severe acute malnutrition (SAM). 2 They report that children with SAM have a mortality risk 11.6 times higher than well-nourished children. They also found that the educational attainment of parents, malignancy, and hypovolemic shock were significant predictors of mortality.
 Other studies have likewise mentioned the negative effects of undernutrition among children.3 They also mentioned increased risk for mortality and morbidity, substantial cognitive deficits in school-age children, grade repetition, and a reduced likelihood of employment in formal sectors in adulthood that may result in lower earning potential. Among school children in the Philippines, less diverse diet, food insecurity, residing in urban areas, being older and male, being part of a larger household, and having parents engaged in fishing or farming can be additional determinants. 4,5 Malnutrition among children, including overweight, is also a significant contributor to other diseases that affects them.6
 In their article, Pabustan-Calleja et al. reported that among the various demographic factors, only the paternal and maternal educational attainment were significantly associated with mortality. Low paternal educational attainment was significantly associated with mortality 21 times compared to high paternal educational attainment while low maternal educational attainment was significantly associated with mortality 51 times as compared to those with high maternal educational attainment.2 These highlight the potential of improving survival outcomes in SAM by improving education.
 The relationship between education (or lack thereof ) and undernutrition is a reciprocal one. Lack of education can lead to undernutrition directly due to poor food choices among those who may not know better. Indirectly, lack of education can lead to lower socio-economic status (SES) contributing to food insecurity and eventually undernutrition. 7 That our educational system leaves much to be desired is indicated by the latest reports. The Philippines has just recently been ranked 77th out of 81 countries in the student assessment conducted by the Organization for Economic Cooperation and Development (OECD) for 15-year-old learners. 8 The relationship between SES and academic performance, however, appear to be a more complex one. For example, in the Programme for International Student Assessment (PISA), students from the Philippines performed significantly lower as compared to students from Turkey and Vietnam belonging to similar socio-economic background.9
 Conversely, undernutrition can lead to inferior cognitive performance, lower academic achievements, lower employment status and in the longer term, also lead to lower SES.10,11 Poor economic status has certainly plagued our country as well for the longest time. According to the latest data, the Philippines is ranked 99 th globally in 2023 in terms of GDP per capita (PPP) and 59 th in terms of the Gini coefficient (a measure of economic inequality with higher being better).12
 Addressing undernutrition among children is an urgent public health crisis. Although mortality due to SAM is obviously important, other consequences such as cognitive impairment and subsequent poor SES need to be recognized as these lead to the unending undernutrition. Improving education in our country in all its aspects, and not just on proper nutrition, can just turn out to be the way to sustain a win in the “hunger games”.

TIMSS vs. PISA: what can they tell us about student success?—a comparison of Swedish and Norwegian TIMSS and PISA 2015 results with a focus on school factors

This paper explores the measurement capabilities of the Trends in International Mathematics and Science Study (TIMSS) and the Program for International Student Assessment (PISA) in assessing school factors that influence student performance. We specifically focus on the 2015 assessments of the science performance of eighth graders in Sweden and Norway. This was the latest year when the two assessments were conducted in the same year and science was the major subject area in the PISA assessment, which was essential for maximizing the assessments’ comparability. Using multilevel models, the study identifies common and unique factors across the assessments and investigates the factors that influence student performance at different proficiency levels. The findings highlight the importance of school-level factors, which are significant in both assessments. Moreover, both assessments provide information on overlapping sets of factors that have varying influence on the performance of students with different proficiency levels. Overall, there are limited common factors between TIMSS and PISA. School factors vary between low-performing and high-performing schools, with differing significance in Norway and Sweden. The results indicate that TIMSS and PISA assessments offer complementary information, particularly for low-performing schools. Our findings suggest that different school types may benefit or suffer from distinct school factors. The findings are relevant for both educational professionals and policy-makers.

Defining and measuring financial literacy in the Indian context: a systematic literature review

PurposeThis paper aims to analyse how financial literacy (FL) is conceptualised and operationalised in the Indian context.Design/methodology/approachA systematic literature review (SLR) was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) protocol. Thirty-six articles published between 2010 and 2020 were considered for analysis. The FL conceptualisation was examined based on knowledge, ability, skill, attitude and confidence elements. The FL operationalisation was analysed using the modified version of the Organisation for Economic Co-operation and Development’s (OECD) Programme for International Student Assessment (PISA) 2012 model for organising the domain for an assessment framework.FindingsThe findings indicate that, despite offering operationalisation details of the FL, 13 out of 36 studies did not include a conceptual definition of FL. Of the 23 studies that mentioned a conceptual definition, 87% are primarily focused on the “knowledge” element and only 39% have combined knowledge, ability/skill and attitude elements in defining FL. As in the developed countries, the Indian studies also preferred investment/saving-related contents in their FL measures. The volume of content focusing on the financial landscape is meagre amongst the FL measures used in India and developed countries. The survey instruments of most studies have been designed in the individuals’ context but have failed to measure the extent to which individuals apply the knowledge in performing their day-to-day financial transactions. Further, it was found that 20 out of 36 studies did not convert the FL level of their target groups into a single indicator or operational value.Originality/valueTo the best of our knowledge, this is the first study that explores the FL’s assessment practices in India. Further, this study offers new insights by comparing the contents of FL measures used in Indian studies with those used in developed countries.

How Social Support Affects Resilience in Disadvantaged Students: The Chain-Mediating Roles of School Belonging and Emotional Experience.

This study aims to utilize data from the Programme for International Student Assessment (PISA), 2018, conducted in four provinces and cities in China, to investigate the impact of social support on the resilience of disadvantaged students. It specifically focuses on the chain-mediated effects of school belonging and emotional experiences. To achieve this, the study selected 2997 disadvantaged students as participants and employed path analysis to examine the mediating effects. The results indicate that teacher support, parental support, school belonging, and positive emotional experiences significantly positively influence the resilience of disadvantaged students, while fear of failure significantly negatively affects resilience. Additionally, teacher and parental support indirectly impact the resilience of disadvantaged students through the chain-mediated effects of school belonging and positive emotional experiences. Similarly, teacher and parental support also influence the resilience of disadvantaged students through the chain-mediated effects of school belonging and fear of failure. To enhance the resilience development of disadvantaged students, it is recommended that teachers and parents provide active attention and support to these students. Simultaneously, there should be increased focus on the psychological well-being of disadvantaged students by strengthening their mental health education.